Wireless server based text to speech email

ABSTRACT

An email system for mobile devices, such as cellular phones and PDAs, is disclosed which allows email messages to be played back on the mobile device as voice messages on demand by way of a media player, thus eliminating the need for a unified messaging system. Email messages are received by the mobile device in a known manner. In accordance with an important aspect of the invention, the email messages are identified by the mobile device as they are received. After the message is identified, the mobile device sends the email message in text format to a server for conversion to speech or voice format. After the message is converted to speech format, the server sends the messages back to the user&#39;s mobile device and notifies the user of the email message and then plays the message back to the user through a media player upon demand.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 11/999,334 entitled“WIRELESS SERVER BASED TEXT TO SPEECH EMAIL” and filed on Dec. 4, 2007,now U.S. Pat. No. 8,103,509 which in turn claims priority to and thebenefit of U.S. Patent Application Ser. No. 60/872,862, filed Dec. 5,2006, hereby incorporated by reference.

BACKGROUND

The present invention relates to an email system and more particularlyto an email system for mobile devices, such as Personal DigitalAssistants (PDAs) and cellular phones that enables email messages toplayed back as a voice message by way of a media player.

Many email systems have the capability to have email messages convertedfor playback as a voice message. An example of such a system isdisclosed in US Patent Application Publication No. US 2007/0174396 A1,entitled: “Email Text-to Speech Conversion in Sender's Voice.” Inaddition to converting email messages to voice messages, the systemdisclosed in the '396 publication is also able to play the message backin the voice of the sender. As illustrated and described in the '396publication, the system disclosed therein includes a unified messagingsystem and a user device that includes a text-to-speech engine. Such aconfiguration precludes the use of the email message to voice messagefeature on mobile devices, such as personal digital assistants (PDAs)and cellular phones since such mobile devices do not normally have theprocessing capabilities for handling a text to speech engine. Moreover,as mentioned above, the system disclosed in the '396 publicationrequires a unified messaging system to operate. As such, the system isnot available for use in applications in systems which do not include aunified messaging system.

Thus, there is a need for a relatively less complex system that allowsemail text messages to be converted to voice messages and played back bymobile devices, such as PDAs and cellular phones.

SUMMARY

Briefly, the present invention relates to an email system for mobiledevices, such as PDAs and cellular phones, which allows email messagesto be identified as they are received by a user's mobile device and thensends the email messages to a server for conversion to an audible formatand subsequently sends the messages back to the user's mobile device;notifying the user of the email message and then playing the messageback to the user through a media player upon demand. As such, the systemin accordance with the present invention provides enhanced functionalityfor PDAs and cellular phones and at the same time is considerably lesscomplicated and thus less expensive than known text to speech emailplayback systems.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of theinvention will be apparent from the following more particulardescription of preferred embodiments herein as illustrated in theaccompanying drawings in which like reference characters refer to thesame parts throughout the different views. The drawings are notnecessarily to scale, with emphasis instead being placed uponillustrating the embodiments, principles and concepts.

FIG. 1 is a block diagram of the system in accordance with the presentinvention.

FIG. 2 is a data flow diagram of the system illustrated in FIG. 1.

DETAILED DESCRIPTION

The present invention relates to an email system for mobile devices,such as cellular phones and PDAs, which allows email messages sent by anemail client to be played back on the mobile device as voice messages ondemand by way of a media player, without the need for a unifiedmessaging system. As will be discussed in more detail below, emailmessages are received by the mobile device in a known manner. Inaccordance with an important aspect of the invention, the email messagesare identified by the mobile device as they are received. After themessage is identified, the mobile device sends the email message in textformat to a server for conversion to speech or voice format. After themessage is converted to speech format, the server sends the messagesback to the user's mobile device as an audio file or alternativelystores the message at the server. In both embodiments, the servernotifies the user of the email message and then plays the correspondingvoice message back to the user by way of a conventional media playerupon demand.

As illustrated in FIG. 1, the system in accordance with the presentinvention, generally identified with the reference numeral 20, includesa mobile device 22 and a server 24. An email client 23 is shown asoriginating an email. The email client 23 does not form a part of thepresent invention. A client side software component resides on a mobiledevice 22, such as a cell phone or similar mobile wireless device. Aserver side software component resides on the server 24. The mobiledevice 22 and the server 24 include transceivers (not shown), whichenable the devices to engage in bi-directional communication over awireless communication link 25.

Client Side Software Component

The client side component can be written in languages that the clientside hardware can support, such as, C++, Symbian, Java, Linux and thelike. The client side component consists of an application interfacethat may include various user configurable features, such as the abilityto enable or disable the system. This allows the user to be selective asto when the application is listening for the email messages. Additionaluser configurable features may include the volume of the audio playback,as well as features such as the voice for the playback, for example,male, female, with and without regional dialects, or the data to beplayed (e.g. From and Subject; All Fields; Subject and Body, and thelike). The system may also include a language feature to enable the userto have the message converted to another language and in the voice of amale or female. The implementation of such features are well within theordinary skill in the art. The client side configuration system may alsobe configured to automatically send data back to a subscription orbilling application that may be included as part of the suite of modulesthat form the server software component to manage active subscriptions.

In addition to the configuration utility, the client side softwarecomponent runs a background process in the form of a listenerapplication or a service that waits for incoming messages. When thelistener receives a notification for a message, it opens the message,such as an SMS, PIN email or standard Email and retrieves the headerinformation to the message to obtain the Sender information, the Subjectline and the Body of the Message. The listener application combines thisdata with a device-specific identifier, for example, the phone number ofthe client device. The information is then automatically sent to theserver 24.

The client side software component in the mobile device 22 determines ifthe data has been sent correctly to the server 24. If it has, the mobiledevice 22 waits for a response to tell it to pickup the speech data inthe device-specific format. When the server side component tells theclient side software component that the speech data is available, theclient side software component retrieves the data from the server 24.Alternatively, the server software component automatically sends thespeech data back to the mobile device 22. In either event, the mobiledevice 22 may optionally append a notification tone any time a convertedmessage is received from the server, if requested, and play the messageback through, for example, the external speaker, the internal handsetspeaker or via a Bluetooth or wired headset coupled to the mobile device22.

The client side software component may additionally provide visualnotification of the converted email. For example, the client sideapplication may include an on demand feature wherein the user chooses anemail for playback from a menu item. Alternatively, the mobile deviceuser may elect to have only selected emails for conversion to speech. Insuch an embodiment, emails are received as text messages by the mobiledevice in a normal manner. In this embodiment, all emails are notautomatically converted to speech. Rather, the system is configured toenable a user to select individual messages for conversion to speech.For example, a “Narrate this Message” menu item may be provided and whenit is selected the client side component follows the same process asdescribed above in connection with the client side listener applicationand sends the message for playback.

Server Side Software Component

The server side software component is generally shown within the dashedbox 24 (FIG. 2). The server side software component includes a datalistener application 26, a context engine 28, a text-to-speech engine 30and a storage medium 32.

The data listener application 26 receives email text data from theclient side application as discussed above. The data listenerapplication 26 transfers the email text data from the mobile device 22client side application to the context engine for filtering, asdiscussed below. The data listener application 26 may also beresponsible for managing subscriptions and validating users and userrequests, as discussed above. The context engine 28 “normalizes” thedata in the email received from the mobile device 22. For example, thecontext engine 28 may be used to convert email addresses and forwardedtext and previous reply email text into either normalized audible dataor leave the data out. In addition, the context engine 28 may includevarious filters for filtering various types of text. For example, afilter may be provided to remove characters that can not be converted toan equivalent speech component, such as dashes, asterisks, punctuation,non-standard characters and lines for reply detail. The context engine28 may also be configured to filter email information that is notrelevant to the user, for example, abbreviations, such as, Re, forward(FWD), attachment information and the like. As well as censored content.Text filters are well known in the art, for example as generallydescribed in US Patent Application Publication No. US 200710233861 A1,hereby incorporated by reference.

Once the context engine 28 has completed normalizing the text dataincluded in the email from the mobile device 22, the normalized textdata is passed it to the Text-to-speech (“TTS”) engine 30. Such TTSengines are well known in the art. The TTS engine 30 can then beinstructed to select a particular TTS voice, for example, male or femalevoice, or virtually any voice including the voices discussed above, toconvert the text to, as selected by the user, as discussed above, andpassed to the server side software component for this purpose. Theserver side software component can also be instructed to convert textfrom one language to another prior to running the data through the TTSengine. This allows for individual emails to be rendered in a languagenative to the user even if it comes from a user speaking a differentlanguage.

Application Example

An application example is illustrated in FIG. 2, which is an exemplarydata flow diagram in accordance with the present invention. The exampleis not intended in any way to otherwise limit the scope of thisinvention. Referring to FIG. 2, an email is sent to a mobile deviceuser. The email in text form is initially received by the server 24 in aconventional manner and is automatically forwarded to the listeningapplication on the mobile device 22, for example, by way of a wirelesscommunication link 34. As mentioned above, the listening application onthe mobile device 22 opens the email and retrieves the headerinformation and combines certain header information with device specificidentifier for the mobile device 22, such as, the telephone number ofthe mobile device 22, and automatically returns this information to theserver 24. The listener application 26 on the server 24 performsvalidation on the connection, the connection data and the userinformation being transferred. Once the data is validated, it is handedoff to the context engine 28 for normalization. After the text data isnormalized, it is directed to the TTS engine 30 where it is converted tospeech data and converted to a conventional audio file format, such as aWAV, MP3 or other audio file format. Once the data is converted to anaudio file, it is loaded into the file folder 32, located at the server24 or optionally at the mobile device 22. The server 24 also notifiesthe mobile device 22 that the audio file is available for playback. Theuser can then listen to the audio file by way of a conventional mediaplayer in the mobile device 22 through the speaker, headset unit orBluetooth device coupled to the mobile device 22.

Obviously, many modifications and variations of the present inventionare possible in light of the above teachings. Thus, it is to beunderstood that, within the scope of the appended claims, the inventionmay be practiced otherwise than as specifically described above.

1. An email system comprising: a server that receives an email directedto a mobile device, the email including text, the server converting thetext in the email into an audio file, the server being in communicationwith the mobile device over a wireless link, the server executing thesteps comprising: receiving a request from a client application of themobile device to convert email text into an audio file; receiving, atthe server via a data listener application, email text data from theclient application of a mobile device; normalizing the email text datavia a context engine, normalizing including leaving out predeterminedportions of the email text data; executing a text-to-speech engine thatconverts normalized email text data into an audio file of recordedspeech; and transmitting the audio file to the mobile device, the mobiledevice having a media player that plays back the audio file as spokentext on the mobile device.
 2. The email system of claim 1, whereinnormalizing the email text data via the context engine includes leavingout email address text and previous reply text from the audio file. 3.The email system of claim 2, further comprising: an interface to theserver that performs a conversion to an audio format, the server being astandalone node that couples with the mobile device, the server avoidingthe need for a unified messaging system having multiple users recognizedby the system; wherein normalizing the email text data via the contextengine includes filtering the email text data such that dashes,asterisks, and punctuation are removed from the email text data prior tocreating the audio file; and wherein the server provides notification tothe mobile device that the audio file is available for retrieval by themobile device for playback on the media player.
 4. The email system ofclaim 3: wherein the server executes a further step that produces theaudio file in accordance with a predetermined selection of gender ofvoice, the selection of gender received via the mobile device; andwherein normalizing the email text data via the context engine includes:converting normalized email text data from a first natural language to asecond natural language; and converting email text from the secondnatural language, via the text-to-speech engine, to recorded speech. 5.The email system of claim 1, wherein normalizing the email text data viathe context engine includes filtering the email text data such thatdashes, asterisks, and punctuation are removed from the email text dataprior to creating the audio file.
 6. The email system of claim 1,wherein normalizing the email text data via the context engine includes:converting normalized email text data from a first natural language to asecond natural language; and converting email text from the secondnatural language, via the text-to-speech engine, to recorded speech. 7.The email system of claim 1, wherein the server executes a further stepthat produces the audio file in accordance with a predeterminedselection of gender of voice, the selection of gender received via themobile device.
 8. The email system of claim 1 further comprising aninterface to the server that performs a conversion to an audio format,the server being a standalone node that couples with the mobile device,the server avoiding the need for a unified messaging system havingmultiple users recognized by the system.
 9. The email system of claim 8wherein the server produces the audio file in a format selected from agroup consisting of: WAV and MP3.
 10. The email system of claim 1wherein the server sends the audio file to the mobile device using thedevice specific identifier.
 11. The email system of claim 1, wherein theserver provides notification to the mobile device that the audio file isavailable for retrieval by the mobile device for playback on the mediaplayer.
 12. An email system comprising: a mobile device, the mobiledevice being in communication with a server over a wireless link, themobile device including a media player that plays an audio file from theserver as a voice message on the mobile device, the mobile deviceexecuting the steps comprising: executing a listening application thatidentifies email messages, the email messages being text communications;receiving an email message at the mobile device; retrieving a messagebody from the email message; retrieving header information from theemail message; retrieving a device specific identifier indicative of themobile device; sending the message body, header information, and devicespecific identifier to the server, the server converting text from theemail message into an audio file of recorded speech; receiving, from theserver, the audio file; and playing the audio file via a speaker of themobile device.
 13. The email system of claim 12, wherein the mobiledevice is a cellular phone.
 14. The email system of claim 12, whereinthe mobile device is a personal digital assistant.
 15. The email systemof claim 12, wherein playing the audio file via the speaker of themobile device is executed in response to receiving a selection thatindicates which fields of an email message to play.
 16. The email systemof claim 12 wherein the mobile device further comprises an enabledstate, the enabled state responsive to user input that activates thelistening for an incoming text message, and selectively performing theconversion based on the state.
 17. The email system of claim 12 furthercomprising an interface that sends data indicative of the conversion toa subscription application, the subscription application correlating theconversion to a particular user based on a fee arrangement.
 18. Theemail system of claim 12, wherein playback of the audio file by themobile device includes playing a spoken version of the email text in asecond language.
 19. The email system of claim 12, wherein the mobiledevice produces an audio tone that indicates receipt of the audio filefrom the server.
 20. The email system of claim 12, wherein clientsoftware in the mobile device provides visual notification indicatingavailability of the audio file from the server.